LAN-HDR: Luminance-based Alignment Network for High Dynamic Range Video Reconstruction

As demands for high-quality videos continue to rise, high-resolution and high-dynamic range (HDR) imaging techniques are drawing attention. To generate an HDR video from low dynamic range (LDR) images, one of the critical steps is the motion compensation between LDR frames, for which most existing works employed the optical flow algorithm. However, these methods suffer from flow estimation errors when saturation or complicated motions exist. In this paper, we propose an end-to-end HDR video composition framework, which aligns LDR frames in the feature space and then merges aligned features into an HDR frame, without relying on pixel-domain optical flow. Specifically, we propose a luminance-based alignment network for HDR (LAN-HDR) consisting of an alignment module and a hallucination module. The alignment module aligns a frame to the adjacent reference by evaluating luminance-based attention, excluding color information. The hallucination module generates sharp details, especially for washed-out areas due to saturation. The aligned and hallucinated features are then blended adaptively to complement each other. Finally, we merge the features to generate a final HDR frame. In training, we adopt a temporal loss, in addition to frame reconstruction losses, to enhance temporal consistency and thus reduce flickering. Extensive experiments demonstrate that our method performs better or comparable to state-of-the-art methods on several benchmarks.Comment: ICCV 202

High Dynamic Range Imaging of Dynamic Scenes with Saturation Compensation but without Explicit Motion Compensation

© 2022 IEEE.High dynamic range (HDR) imaging is a highly challenging task since a large amount of information is lost due to the limitations of camera sensors. For HDR imaging, some methods capture multiple low dynamic range (LDR) images with altering exposures to aggregate more information. However, these approaches introduce ghosting artifacts when significant inter-frame motions are present. Moreover, although multi-exposure images are given, we have little information in severely over-exposed areas. Most existing methods focus on motion compensation, i.e., alignment of multiple LDR shots to reduce the ghosting artifacts, but they still produce unsatisfying results. These methods also rather overlook the need to restore the saturated areas. In this paper, we generate well-aligned multi-exposure features by reformulating a motion alignment problem into a simple brightness adjustment problem. In addition, we propose a coarse-to-fine merging strategy with explicit saturation compensation. The saturated areas are reconstructed with similar well-exposed content using adaptive contextual attention. We demonstrate that our method outperforms the state-of-the-art methods regarding qualitative and quantitative evaluations.N

DSBN: Detail-Structure Blending Network for Video Super-Resolution

Video super-resolution (VSR) aims to generate a high-resolution (HR) video by reconstructing each frame from the corresponding low-resolution (LR) frame and its neighboring ones. However, it is challenging to generate a clean HR frame due to displacement between the frames. To alleviate this problem, most existing approaches explicitly aligned neighboring frames to the reference. However, these methods tend to generate noticeable artifacts in the aligned results. In this paper, we propose a detail-structure blending network (DSBN), which structurally aligns the adjacent frames by blending the frames in deep-feature-domain. The DSBN extracts deep features from the reference frame and each neighboring frame separately, and blends the extracted detail and structure information to obtain the well-aligned results. Afterward, a simple reconstruction network generates a final HR frame using the aligned frames. Experimental results on the benchmark datasets demonstrate that our method produces high-quality HR results even from videos with non-rigid motions and occlusions.N

Kernel Prediction Network for Detail-Preserving High Dynamic Range Imaging

Generating a high dynamic range (HDR) image from multiple exposure images is challenging in the presence of significant motions, which usually causes ghosting artifacts. To alleviate this problem, previous methods explicitly align the input images before merging the controlled exposure images. Although recent works try to learn the HDR imaging process using a convolutional neural network (CNN), they still suffer from ghosting or blurring artifacts and missing details in extremely under/overexposed areas. In this paper, we propose an end-to-end framework for detail-preserving HDR imaging of dynamic scenes. Our method employs a kernel prediction network and produces per-pixel kernels to fully utilize every pixel and its neighborhood in input images for the successful alignment. After applying the kernels to the input images, we generate a final HDR image using a simple merging network. The proposed framework is an end-to-end trainable method without any preprocessing, which not only avoids ghosting or blurring artifacts but also hallucinates fine details effectively. We demonstrate that our method provides comparable results to the state-of-the-art methods regarding qualitative and quantitative evaluations.N